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Claims 




1 • A method for processing frames of streaming data through modules in a digital 
computer, comprising: 

constructing a pipe as a connected group of multiple opes of the modules, at least 
one of the modules being a restructuring module; X 

allocating composite frames having predefm^d subframes; 

transporting the streaming data througKaifferent ones of the modules in the group 
in different ones of the subframes; / 

restructuring the data amongst least some of the subframes in the restructuring 
module. / 

2. A computer readableinedium bearing instructions and data for causing a digital 
computer to execute ine method of claim 1. 

3. The method of claim 1 where the composite frame is a physical frame in a memory. 

4. The^method of claim 3 where the subframes are virtual frames defined in the same 
memory as the physical frame. 

5. The method of claim 1 further comprising assigning an allocator for the composite 
frames to one of the modules in the pipe. 



'6. The method of claim 5 where^the allocator is assigned to the farthest upstream 
restructuring module in the^fipe. 
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7. The method of claim 5 wher^th5allocator is assigned to the farthest downstream 
restructuring module itrfh^pipe. 

8. The method of claim 1 further comprising constructing a frame nesting tree specifying 
how the data is to be restructured in the restructuring module. 



9. The method of claim 1 further coirmju 

repeating th^-abov^st^ps for further pipes including further modules in the 
computer; 

jokfing the pipes together into a graph. 



1^. The method of claim 1 where transporting the streaming data includes issuing a 
control transaction to the restructuring module only when all of the subframes processed 
by that module become available. 




1 1. A method for processing frames of streaming data througfefmultiple modules disposed 
in a pipe including at least one restructuring module in a^tfgital computer, comprising: 

allocating a composite frame having multiple subframes; 

performing operations upon the subfhjmes in any of the modules sourcing data to 
the restructuring module; 

after completion of the operations for all of the subframes sourcing data to the 
restructuring module, issuing a comrol transaction to the restructuring module; 

performing operations/upon the subframes sourced to the restructuring module in 
response to the control transaction. 



12. A computer readable medium bearing instructions and data for causing a digital 
computer to execute the method of claim 11. 
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13. The method of claim 1 1 where allocating the cpm^osite frame includes constructing a 
frame control table having an entry for eapfrlnodule in the pipe and a flag for each of the 
modules indicating whether a partietflar module has completed an operation upon the 
frame. 



14. The methodof claim 13 further comprising setting one of the flags whenever a 
module h^s completed an operation upon a frame. 

A, iq 

\L The method of claim J4 further including constructing an offset table specifying the 
structure of the sub frames within the composite frame. 

Lo. The method of claim ]A further comprising constructing a pipe control table 
specifying the structure of the modules in the pipe. 

/ J7. The method of claim \Jo where the pipe control table has an entry for at least some of 
the modules in the pipe, and where each entry specifies which other module or modules 
source data to one of the modules in the pipe. 



yi. The method of claim 1# where the pipe control table has an entry for each module in 
the pipe that sources data from outside the pipe. 



19. The method of claim 11/further comprising repeating the above steps overlapped in 
time such that a plurality Df composite frames circulate within the pipe concurrently. 

f). The method of claim j^further comprising constructing a separate frame control 
table for each of the composite frames. 
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21. A method for processing frames of streaming data through mcyitffes including 
multiple restructuring modules in a digital computer, com^p^mg: 

constructing a pipe as a connected group oftja^modules including the 
restructuring modules; 

assigning a single allocator to ope'of the modules; 

allocating composite frames having predefined subframes associated with 
respective ones of the restrucftuing modules; 

transporting the streaming data through different ones of the modules in the group 
in different ones of tire subframes; 

restructuring the data among the subframes in the restructuring modules. 



22. A computer readable medium bearing instructions and data for causing a digital 
computer to execute the method of claim 2 1 . 
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23. The method of claim j21 further comprising constructing a frame nesting tree 
specifying how the data is restructured by the restructuring module. 

The method of claim J2!i further comprising collecting a constant-offset flag for each 
module. 

y£. The method of claim 24 further comprising collecting an offset value for any module 
whose constant-flag is set. 
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jk. The method of claim ^4 further comprising constructing an offset table specifying 
relationships of the subframes to the composite frame. 
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27. The method of claim 24 further comprising specifying memory sizes for each of the 
subframes within the composite frame. 

v \ ^ 

2%. The method of claim $i where a plurality of the restructuring modules are of a single 
type in a cascade in the pipe. 

29. The method of claim 2« where the allocator is assigned to a particular module m 
response to the type of cascaded modules. 

10. The method of claim 28 where the plurality of cascaded modules are mixers, and 
where the allocator is assigned to a downstream one of the cascaded mixers. 

."7° ^7 

y(. The method of claim ^ where the plurality of cascaded modules are splitters, and 
where the allocator is assigned to an upstream one of the cascaded splitters. 



1? 



The method of claim ^\ where transporting the streaming data includes issuing 
control transactions to the restructuring modules only when all of the subframes 
processed by respective ones of the modules become available. 



33. A method for processing frames of stemming data through modules including 
multiple restructuring modules ccmn^cted in a pipe in a digital computer, comprising: 

allocating a composite frame having multiple subframes for different ones of the 
restructuring module?^ 

perfomring data-sourcing operations upon certain of the subframes in source ones 
of the mojdules; 
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when each one of the data-sourcing operations ha^jtxrfnpleted, determining 
whether one of the restructuring modules has all j)£t1ie subframes required for it to 
perform an operation; 

if so, issuing a control transaction to the one restructuring module; 

performing an opej^tion in the one restructuring modules after receiving a control 
transaction. 

34. A computer readable medium bearing instructions and data for causing a digital 
compu^r to execute the method of claim 33. 

*ft *>y 

36. The method of claim 35 where the source ones of the modules are those receiving 
data from outside the pipe. 
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3jb. The method of claim ^ where a plurality of the restructuring modules are mixers. 



yi. The method of claim So where a plurality of the mixers are cascaded m the pipe. 



y&. The method of claim ft further comprising: 

when each one of the data-sourcing operations has completed, determining 
whether others of the restructuring modules have all of the subframes required for them 
to perform operations; 

if so, issuing control transactions to the other restructuring modules; 

performing operations in the other restructuring modules after receiving the 
control transactions. 




9. The method 01 claim 33 further comprising: 
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storing a separate completion flag^fSr each of the modules in the pipe; 

setting one of the compldion flags when a corresponding one of the modules has 
completed an operatiorj^on the frame. 

% i 

40. The method of claim 1/9 where the determining step composes: 

reading a table listing all of the other modules that source data to the one 
restructuring module; 

determining whether a completion flag for all of the other modules has been set. 

41. The method of claim 33 further comprising repeating the above steps overlapped in 
time such that a plurality^di composite frames circulate within the pipe concurrently. 

42. The method of claim 41 further comprising constructing a separate frame control 
table for each of the composite frames. 



43. A data structure for processing stp^aming data through multiple restructuring 
processing modules in a graph^romprising: 

a single comppsite frame physically allocated in a memory; 

a plurality of virtual sub frames allocated within the composite frame. 



44. The structure of claim\3 where different ones of the sub frames contain different 
types of data. 



45. The structure of claim 44 where one 





ata is video data. 



46. The structure of claim 44 where another type of data is audio data. 
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55. The structure of claim !54 where the frame control table includes a column for holding 
flags indicating that each of the modules has completed processing the frame. 




56. The structure of claim 53 further comprising a separate frame control table for each 
frame circulating in the pipe at a particular time. 
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(57. A computer system for processing streaming data, cofliprising: 

a plurality of modules for processing the stj^aming data, at least some of which 
are restructuring; 

a plurality of memory managers f#f allocating composite frames containing 
sub frames for containing streaming dma; 

a flow manager for conducting a graph having at least one pipe including a 
plurality of the restracturing/ftodules, and for assigning one of the memory managers to 
the pipe. 

58. The system o^claim 57 further comprising a memory for storing the composite 
frames. 



59. Tha^system of claim 58 where the subframes are allocated within the composite 
frames in the memory. 

£0. The system of claim p% further including a processor. 

64. The system of claim 60 where the processor implements one or more of the 
restructuring modules. 
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$1. The system of claim^) further including an input/output system. 

6^. The system of claim^Z where the input/output system implements one or more of the 
modules. 

6ji. The system of claim yl further comprising a pipe manager for constructing 
representations of the structures of the composite frames. 

s° ^ 

6j(. The system of claim £4 where one of the representations specifies locations of the 
subframes with respect to the composite frame. 

66. The system of claim 0 where one of the representations specifies relationships of the 
modules within the pipe. 

6jf. The system of claim ^/l further including a control manager for transporting the 
streaming data through the modules in various ones of the subframes. 



68. The system of claim 67 where the control manager issues a control transaction to one 
of the restructuring module for initiating processing^ a subframe therein only when all 
of the subframes processed by that module bpeome available. 

69. A computer system for proce^ing streaming data, comprising: 

a plurality of module/disposed in a pipe for processing the streaming data, at 
least some of the modulegi)eing restructuring; 

a plurality of/memory managers for allocating composite frames containing 
subframes for comaining streaming data, different ones of the subframes being associated 
with different^ones of the modules; 
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a control manager for issuing control transactions for initiating processing 
operations in the modujj 

Ifl. The system of claim 69 where the control manager issues one of the control 
transactions to a particular module only when certain of the sub frames associated with 
that module become available. 
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74. The system of claim JO where the certain subframes include all of the sub frames that 
source data to the particular module. 

12. The system of claim 6p where certain of the modules receive data from outside the 
pipe. 

p. The system of claim 72 where the control manager issues a control transaction to the 
certain modules when one of the composite frames has been allocated. 

74. The system of claim £9 where at least some of the restructuring modules are mixers 
for combining multiple ones of the subframes. 
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lj>. The system of claim J4 where a plurality of the mixers are cascaded in the pipe. 



. The system of claim ^ where the control manager issues one of the control 
transactions to any of the mixers only when all of the subframes combined by that mixer 
become available. 




77. The system of claim 69 fijrtfier comprising a memory for storing the composite 
frames. 
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78. The system of claimj^^ the processor implementing one or more of the 
restructuring modules. 

^ 9 

19. The system of claim 78 further including an input/output system implementing one or 
more of the modules. 



80. A computer readable medium bearing instructions and data ^6r causing a digital 
computer to execute a method for processing frames of streaming data through modules 
in a digital computer, the method comprising: 

constructing a pipe as a connected group of i^fultiple ones of the modules, at least 
one of the modules being restructuring; 

allocating composite frames having undefined subframes; 

transporting the streaming data through different ones of the modules in the group 
in different ones of the subframes; 



restructuring the data among at least some of the subframes in the restructuring 



module. 



81 . A computer readableonedium bearing instructions and data for causing a digital 
computer to execute a/method for processing frames of streaming data through multiple 
modules disposed hi a pipe including at least one restructuring module in a digital 
computer, the mefliod comprising: 

allocating a composite frame having multiple subframes; 

performing operations upon the subframes in any of the modules sourcing data to 
the restructuring module; 

after completion of the operations for all of the subframes sourcing data to the 
restructuring module, issuing a control transaction to the restructuring module; 
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47. The structure of claim 43 further comprising an offset table for locating the subframes 
with respect to the composite frame. 

48. The structure of claim 4u where the offset table includes an entry for each of the 
subframes and a column for Specifying the locations of the subframes within the 
composite frame. 



49. The structure of claim 43 further comprising a pipe control table representing the 
structure of a pipe of modules for Wocessing^streaming data in the data structure. 




50. The structure of claim 49 where t^e pipe £mrtr6I table includes an entry for each of 
the modules in the pipe. 



51. The structure of claim 50 where the pipe control table includes a source column for 
listing other modules that source data to the modules in each of the entries. 

52. The structure of claim 50 where the pipe control table includes a start section listing 
those modules that source data from outside the pipe. 

53. The structure of claim 43 further comprising a frame control table for tracking the 
status of the frame with respect to the processing modules. 



54. The structure of claim 53 where the frame control tal\le includes an entry for each of 
the modules in the pipe. 
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performing operations upon the su^frtfmes sourced to the restructuring module 
response to the control transaction^ 



31 



